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METHODS FOR SMOOTHING EXPECTANCY TABLES 


APPLIED TO THE PREDICTION OF SUCCESS IN COLLEGE 
“ABSTRACT 
vl 
& \ 
Six methods for smoothing expectancy tables were compared using data 
for entering students at 86 colleges and universities. Linear 


regression analyses were applied to ACT scores and high school grades’ 

to obtain predicted first term grade point averages (FGPA's)-for students 
entering each institution in 1969-70. Expectancy tables were constructed 
_ for each institution using the relative frequency with which students 

vd each predicted FGPA intaevad obtained FGPA's of "B or better" and "C or 
better". The methods used to smooth the 1969-70 expectancy Sditnnneen 
compared on how accurately the smoothed tables corresponded to the 1971-72 
and 1972-73 seanied data. The smoothed tables vere substantially more 
‘accurate than the 1969-70 observed data on which they were based. Two 


maximum likelihood methods yielded the most accurate tables. Estimated 


relative frequencies were more. accurate for the "B or better" level than 


the "C or better" level, 


a 


METHODS FOR SMOOTHING EXPECTANCY TABLES 


APPLYED TO THE PREDICTION OF SUCCESS IN COLLEGE 


In many situations involving the selnction of qualified applicants, 
the measure of ‘primary interest is not a predicted gtbte on some criterion, 
but the probability that an individual will obtain a criterion score eqfal 
to or greater than some pre-defined level.:,A college or university, for 
example, may be more interested in estimating the number of applicants 
with a given predicted grade-point average (PGPA) which “a be packed 


to achieve an overall grade of C or better than in knowing each applicant's 


PGPA. 


Although a expectancy table may be formed irrespective of the re- 
lationship between actual and predicted grade point averages, the concern | 
in this paper is with the large class of bivariate relationships for which 
the first term grade point average (FGPA) and the predicted grade point 
average (PGPA) have a monotonic. nandecreasing relationship. That is, the 
expected value of FGPA for any PGPA value is not less than the expected 
value of FGPA for any higher PGPA. This allows for a wide variety of bi- 
variate distributions including all positive linear and certain nonlinear 


Be 
relationships. Although monotonic nondecreasing relationships may not occur 


* / 
in special circumstahces involving students with extremely low or high PGPA's 


‘ 


(Novick and Jackson, 1974), this assumption seems generally justifiable 


throughout the entire range of grade point averages for most colleges. - 


¥ 


. Note: The authors are grateful to Dr. E. James Maxey and the American 
College Testing Program for their support and assistance. 
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To construct an expectancy table, two measures must be available for some 
large nupber of observations (the experience pool) representative of the 
oapulacion for which the prediction is to be made. dypicadiy, when evades 
ing success in college, a previous freshman class is used 4a: the experience 
pool. If the number of observations in the experience pool is sufficiently 


large, the expectancy table may be constricted directly from the observed 


Sy 


data. One advantage of this method, as compared to others discussed below, 


is that. it is relatively easy and requires only that individuals in the 


experience pool be assigned to the appropriate .cell of the table representing 


a combination of PGPA and FGPA values. The cefl frequencies are usually con- 
= ‘ 
verted to relative frequencies or cumulative relative frequencies which are ae 


used to make predictions. 


There are two potential }imitations in this method of constructing 


. 


expectancy tables. The; first is that there are no absolute rules to determine 
what constitutes a "sufficiently large" number of observations in the exper- 
ience pool. The second is that reversals will often occur. That i8, the 


data reported in the table will be inconsistent with the.assumptions about the 


relationship between FGPA and PGPA. The usual explanation for these reversals 


* 


is that they are due to the sampling fluctuations of cell estimates 


based on few observations. 


‘ 


Methods for Smoothing Expectancy Tables > 


\ 
r] 
Expectancy tables can be derived ,using statistical theory rather than 


basing the table solely on the observed frequencies. The advantage of using 
a theoretical basis to smooth an expectancy table constructed from observed 
data is that it is possible to base estimated relative frequencies pot only 


upon the observations in each cell but also upon the observations in adjacent 


| 


t 


o 


Lays 


cells. Such a procedure woul be expected to Neve: at effect similar to in- 
creasing the number’ of observations in a particular cell and would temove 
revarnele Semele in cell entries which née accurstely describe the under- 
lying relationship. 

She Linieation in using a theoretical approach is that scnmhat more 
statistical woplidatieacion is required. A second limitation is that actual 
dhatucterieticn of the relationship between the variables would be lost if 
tables were derived using an incorre¢t theoretical basis (Schraeder, 1965). 
In case of act iéue enade point averages, emaaic this appears unlikely. 

The six smoothing methods investigated ‘included a linear regression 
procedure similar to that used by the American College Testing (ACT) Program 
(ACT, 1971), an isotonic regtession method attributed to Brunk (Ayer, Brunk, 2 
Ewing, Reid and Silverman,. 1955), and four methods previous ly used primarily 
in bioassay problems. “Two of the bioassay methods were iterative maximum 
aikedtheos procedures (Finney, 1971); one requiring a normal and the other a 
Logistic transformation of the observed relative frequencies. The other two 
bioassay methods were noniterative minimum x? procedures. One required a 
normal transformation (Bedkdon, 1955) and the other a logistic transformation 
(Berkson, 1944) of the observed relative frequencies. Perrin (1974, 1975) 
provides a complete description of the methods as applied in this study. 

The linear regression method assumes a linear relationship between 
PGP and FGPA and normal,homoscedastic conditional distributions of FGPA 
at each PGPA level. Least Squares techniques provide the estimated means 
and standard deviations of the conditional FGPA distributions. These statistics 
are used in conjunction with the FGPA level of interest (e.g., 2.0 for C or 
bettgr) to.compute a standard deviate for each PGPA level. These deviates 
are referred to normal probability tables to estimate the relative frequencies 


of attaining each FGPA. 
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The isotonic regression method requires only the observed relative 
frequencies exceeding the chosen FGPA level at each PGPA level. When there 


are no reversals, these observations aré the estimated saeco alae 


st 


When the observed relative frequencies for ‘two adjacent PGPA levels are ‘reversed, 
a weighted avbrage of the two observations is computed and substituted for each 
relative Fxedumcy: This weighted averaging procedure is continued until all 


reversals ate removed. The numbers én this series, some of which are observed 
relative frequencies and some of-which are: weighted aVEEOEOS are the estimated 
relative eraquenctey resulting from the isotonic regression method. Additionally, 
it is necessary to extrapolate or interpolate to obtain aatinates for PGPA 
levels for which there were no ‘observations ins the base year. 

The maxitium likelihood methods used either an ievbcad normal or logistic 
transformation on the observed relative frequencies. The resulting: values are 
iteratively regressed on PCPA until the difference between the. sum of the 


differences between regression estimates for two successive iferations is 
a 


sufficiently small (for this study 10 a Using the inverse of the 


original transformation, the regression estimates from the final iteration 


are transformed into the estimated relative fruduantians 

The minimum ? nathods also depend on tranformiations of the observed | 
relative frequencies, However, the’ use of additional approximations suggested 
by Berkson (1944, 1955) eliminates the need for iteration. Using weights which 
are determined by the form of the transformation, the transformed values are + 
regressed ‘on PGPA: The inverse of the original transformation gn the values 


resulting from the regression yields the estimated relative frequencies. \ 


ry 


Procedures. 
the data for this study watt the records of entering freshman icadints 
"atva sample of iastitutions which had paxticipated in one of the ACT Research 
Services for: the years 1969-70, 1971-72 and 1972-73. (Henceforth, 1969-70 
i * 


will be referred to as the base year and 1971-72 and 1972-73 as, respectively, 


validation year one and two.) The academic year 1970-71 was not included 


because, in the standatd ACT procedures, predictions based on the base year 
hive would not have -been available for most freshman iceetoas in 1970. 

Frem a list of all institutions that had participated in sana Sedeareh 
Service program for all three years, a stratified random sample of 86 institu- 
tions was \drawn. The siete included 22 two-year institutions, bt. twatitovions 
granting only the bachelor's degree, 21 institutions pemabing at least one 
graduate dante other thai the doctorate, and 22 inatitutions granting the 
doctorate (types 1, 2, 3 and 4 respectively). For each student completing 
the first term in one of these schools during the base year or either of the 
validation years, ACT provided the student's four ACT Assessment sibtant scores, 

\ 


average of four self-reported high school grades, first term grade-point average 


(FGPA) at the institution, and sex. 


Prediction of FGPA \ 

Following the standard ACT procedure, regression analyses were conducted 
on the base year data separately by sex if both the numbers of men and women 
completing the first term at an institution exceeded 100. If' either the 
number of] men or women was less than 100, a single regression analysis was 
conducted for that institution. The separate regressions by sex’ were used 


whenever possible because the multiple correlation betweeit’ FGPA and the five 


independent measures is typically higher for women than fqr_men (ACT, 1971). 


‘ ar 


The regression analysis used five, independent variables (four ACT 
Assessment subtest scores and the average of the weleotesorked high school 
grades) and was comparable to that used by ACT in‘its Basic Research Service. 
| The regression weights calculated by this procedure were used to compute a 
PGPA for each student in the base and validation years, Similar analyees 


. tok { 
were conducted using. a nonlinear regression procedure (Perrin, 1974) but are 


not reported since the improvement in predictive accuracy was slight and 


the conclusions concerning methods were similar to those reported here. 
' ‘ 


Constructing and Smoothing Expectancy Tables 


Following the computat}on of PGPA, ores expectancy tables (one for 
each year) were constructed for each ietlcsitin. The base vane table for 
each school was smoothed by application_of the methods independently to each 
column of the table. — Hee, for each base year table; six different "smoothed" : 
, tables were constructed. Bach of these gix tables contained relative 
frequencies, estimated by one of thé methods, of obtaining a FPGA equal to or 


greater than C and B. 


Criterion Measures ‘ ; com Ai 


The criterion of interest was the flegree, to which each set of relative 
frequencies estimated from the base year data corresponded to the relative 


; / 
frequencies observed in each of the validation years. The criterion measure 
/ « . : 


/ 
/ 


. for this study was ~ / =n at ~ 
— — / ; ; 
k -1 ~ \2 |1/2 
=k (py, - 7,) . ; : 
i=1 i 1 4 . \ (1). = 
*. a Fad 
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where -p, is the validation year observed relative frequency, t, is the estimated 


~ 


relative frequency resulting from the application ‘of ‘one of the smoothing methods, 


and k is the number of PGPA levels for which observations were available in the 


validation year. That is, the criterion meagure was! the positiye square root of 
hese squared er?or for k levels of PGPA. A small value forthe criterion 


measure identified the method(s)* producing the smallest: estimation, errors 


across all PGPA levels. Similar results, using two other criterion measures, 


4 
are reported by Perrin (1974). 
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“™\ Analysis of Data- 


‘ 


A four factor mixed analysis of variance (ANOVA) procedure was applied 


to the criterion measure (Lindquist, 1953). Factors in the analysis were 


institutional type (four levels), expectancy table construction method (seven 
levels), FGPA value (two levels), ‘and validation year (two levels). 
Institutional type was considered a "between" effect and all other factors 

were treated as "within" effects. The wit of analysis was school within types. 
One construction method was the observed relative frequencies for the ane year; 
the other six methods were those described above. Each of the methods yielded 
a ctiterion measure for two levels of FGPA ("C or better" and "B or better") 


for each validation year ae each school. 
In addition to the effects enialis tested in such an- analysis, three 

questions were of special interest. First, did the base year observed relative 

frequencies provide estimates which were as accurate ss those provided by the 

six emgothing methods? , Second, was there a difference in accuracy between the 

linear“ regression method presently used by- ACT and the other five smoothing methods? 

Third, was there a difference in Acdutacy between the bioassay methods and the 


simpler isotonic regression method?’ Additional analyses were conducted to answer 


each of these questions using’ pre-planned contrasts. 


10 
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Smoothing Methods 


; The results of the analyses conducted to evaluate the effects of the 
smoothing methods are summarized in Table 1; The Mean square ratio for the, 
main effect of methods exceeded the value of’ the 99th percentile in the 
appropriate F-di8tribution. ‘Additionally, the value of the statistic comparing f 
the criterion measure mean for base year vatuelive frequencies with the average 
of the criterion measure means for the six smoothing methods exceeded the 
appropriate 99th percentile value. nin, deivantée measure means for each method 
appear in Table 26 The results indicated chit /the smoothing methods provided 
more accurate estimates of the eelative: Praqueacies observed in the validation 
years than did unsmoothed base year relative brecuansiad Thus, it is recommended 
that expectancy tables used to sntiaate relative frequencies of success in 


7 


college should be smoothed rather than simply based on observed relative 


pie aa 


a a et ee 


Insert Table 2 about here 


The comparison of the average criterion measure Yor the ldnear wegreasion 
method with the average criterion measure for the other five smoothing methods 
also yielded a statistic greater than the appropriate 99th percentile value. This 
suggests that the method, presently used by ACT-is less accurate than the average 

\ 


“Accuracy of the other smoothing methods. Generally, conclusions and recommen- 


dations based on main effects would not be made when interactions involving 


that factor were observed. In this case, however, examination of simle effects 


® 


at each level of the interacting factors did not contradict the conclusions 
i 


drawn from the main effects for methods. (That is, the interaction was not 


disordinal.) Therefore, it is recommended that one of the other smoothing 


methods be used in preference to the linear’ regression method. 


) 
, i 
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Which of the other smoothing methods should be adopted was not entirely 


clear. However, the maximum likelihood methods provided more accurate estimates 


of the validation years' relative frequencies than did the other smoothing 

| : . 
methods; For institutions without the computational facilities or inclination 
to apply the maximum likelihood methods, the simpler isotonic regression method 


- could be expected to provide estimates nearly as accurate as the more complex 
; 
methods. ‘ 5 : be ‘ 


FGPA Level : 

—_ See: 

Estimation of relative frequencies was more accurate for the "B or better" 
level than for the "C or better" level. The differences between the mean 


criterion measures for the "B or better" level and those for the "C or better" 
- 


level: exceeded the appropriate Scheffé 1% critical difference (Scheffé, 1959). 

: Validation Year \ 
All criterion measure means for validation year one were lower than 

those for validation year two. (That is, accuracy declined over time.) 

However, the magnitude of these differences did not exceed the appropriate 

Scheffé critical atctexancas The criterion measure ine for the FGPA level 

by validation year suggested that most of the difference in predictive accuracy 

between validation years was due to a loss of pecivacy at the "B or better" 

level. Thus, although predictive accuracy was generally lower for the "C or 

\ 


a 
better" level, accuracy at that level declined less between validation 


years * and two than for the "B or better” level. 


Institutional Type 


° . 
The results for the main effect of institutional type and for the simple 


effects of institutional type for each method indicated that ‘the most accurate 


estimation of relative frequencies occurred at type 4 and Type 1 institutions. 


The mean and median sizes of the entering freshman clagses for each of the 
a a a : 
\ Ee 


- wv “ 


, a: 


institutional types were inversely related. to the accuracy indexes. 


These were considered likely to have contributed to the differences in 
; y 


estimation accuracy among the institutional types. That’ is, the smoothing 


methods and base year observed frequeneies yielded the more accurate estimation 


for institutions with the largest entering enrollments. 


Conclusion 


. 


Because of the widespread use of expectancy tables by colleges and 
universities, any improvement in accuracy would be beneficial. Of special 
importance are the uses of expectancy tables for guidance, admissions, and 


? 


planning purposes. Application of the best smoothing method appears likely 


to result insa substantial improvement in the accuracy of the information used 


in individual decisions. Additionally, application of the best smoothing method 
. , a 


would improve estimates of the number of students likely to be dropped for 


scholastic reasons. In any of these situations, more accurate information 


should result in better dectsions. 


. 


war. 
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TABLE 1 
ANALYSIS OF’ VARIANCE 


SUMMARY TABLE 


\ 


Between Schools 


D(Institutional Type) 0.3492 
Error > ‘ 0.0514 


Within Schools 


%  A(Method) ” , 0.3439 
AD(Method x Type) : 0.0085 
Error 0.0022 


B(FGPA Level) ; 1.3277 
BD(FGPA Level x. Type) ; 0.0568 
Error ~— 0.0204 


C(Validation Year) : 0.0274 
CD(Year x Type) 0.0092 
Error 0.0148 


AB(Method x FGPA Level) 0.0209, ° 
ABD(Method x FGPA Level x Type) 0.0009 
Error 0.0008 


_ AC(Mathod x Year) 0.0005 
ACD(Method x Year x Type) 0.0005 
Error < 0.0007 


WBC(FGPA Level x Year) - : 0.0864 
BCD(FGPA Level x Year x Type): 0.0214 
Error ° 0.0121 


ABC(Method x FGPA Level x Type) 0.0003 
- ABCD(Method x FPGA Level x 

Year x Type) “~~ ‘0.0004 

Error 0.0003 


Total 


*p<.01. 


TABLE 2 


CRITERION MEASURE MEANS 


Smoothing Nethod Institut ional Type 


B or Better C or Better 


Linear Regression 

twopani Regression 

Maximum Likelihood (Normal) 
aximum Likelihood (Logistic) 
. Minimum x? (Normal) 
. Minimum x? (Logistic) 


Base Year Relative Frequencies 


All Methods 


